Conventional user-assisted video object selection and segmentation techniques can be divided into two general categories: curve-based and brush-based methods. In curve-based methods, an object boundary is generally represented by a set of smooth curves, such as Bezier curves. A user interface is typically provided to allow a user to precisely place and adjust control points of these curves along the object boundary for accurate selection of the object. Simple interpolation is then applied between two non-adjacent key frames to produce segmentation on intermediate frames. This process is generally known as “Rotoscoping”. Such methods can provide fine control on the selection, and generate high-quality smooth curves with both spatial and temporal stability across frames. However, such methods require the user to manually align the curves on numerous key frames, causing this process to be very time-consuming and labor-intensive.
The brush-based methods generally require the user to paint brush strokes in the interior of the desired object, and negative strokes outside of the object. A segmentation algorithm then calculates an optimal segmentation outline, also referred to herein as a mask, using the brush strokes as hard constraints. The mask is updated progressively to achieve the desired segmentation as the user continues to add more brush strokes. Compared with curve-based methods, this approach requires less amount of user brush strokes, which are not required to be precisely placed, thus significantly simplifying the workflow and shortening the work time. However, these brush-based methods generally produce relatively low quality, non-parameterized boundary curves that are not easily controllable by the user. Moreover, although the brush-based methods account for temporal coherence, their results are inferior in terms of temporal stability, generally containing visually distracting boundary chattering and causing the results to be unusable in video production. Because the users do not have precise control on the resulting curves, it is challenging to correct small errors on the boundaries to further improve the results.